Interactive 3D cursor for use in medical imaging

ABSTRACT

An interactive 3D cursor facilitates selection and manipulation of a three-dimensional volume from a three-dimensional image. The selected volume image may be transparency-adjusted and filtered to remove selected tissues from view. Qualitative and quantitative analysis of tissues in a selected volume may be performed. Location indicators, annotations, and registration markers may be overlaid on selected volume images.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.17/021,548, filed Sep. 15, 2020, which is a Continuation of U.S. patentapplication Ser. No. 15/878,463, filed Jan. 24, 2018, now U.S. Pat. No.10,795,457, which is a Continuation-in-Part of U.S. patent applicationSer. No. 14/877,442, filed Oct. 7, 2015, now U.S. Pat. No. 9,980,691,which is a Continuation-in-Part of U.S. patent application Ser. No.12/176,569, filed Jul. 21, 2008, now U.S. Pat. No. 9,349,183, which is aContinuation-in-Part of U.S. patent application Ser. No. 11/941,578,filed Nov. 16, 2007, now U.S. Pat. No. 8,384,771, which claims thebenefit of and priority under 35 U.S.C. § 119(e) to U.S. PatentApplication No. 60/877,931, filed Dec. 28, 2006, each of which areincorporated herein by reference in their entirety.

TECHNICAL FIELD

Aspects of this disclosure are generally related to human-machineinterfaces, and more particularly to cursors.

BACKGROUND

The typical arrow-shaped cursor presented by a computer operating systemis zero-dimensional. A zero-dimensional cursor designates the locationof a single point in a space such as a two-dimensional window presentedon a monitor. Mouse buttons can be used in combination with movement ofthe cursor to select objects in the two-dimensional space, but at anygiven instant of time a zero-dimensional cursor position designates onlya single point in space.

The current standard for diagnostic radiologists reviewing computedtomography (CT) or magnetic resonance imaging (MRI) studies is aslice-by-slice method. A conventional keyboard, monitor, and mouse witha zero-dimensional cursor are used for manipulating the images. The useof mouse buttons and cursor movement for manipulating the images canbecome burdensome. For example, many images are included in radiologystudies that are performed for the follow up of cancer to determine theresponse to treatment. The ability to recognize and analyze differencesbetween images can be important. As an example, the recent Investigationof Serial Studies to Predict Your Therapeutic Response with Imaging andMolecular Analysis (I-SPY) trial tracked the changes in the tumor overmultiple magnetic resonance imaging (MRI) scans during theadministration of neoadjuvant chemotherapy (NACT). It has been notedthat the phenotypic appearance (e.g., shape, margins) of a tumorcorrelated with the pathologic response to NACT. A more efficient andaccurate interface for manipulating and presenting medical images wouldtherefore have utility.

Known techniques for 3D viewing of medical images are described in U.S.Pat. No. 9,349,183, Method and Apparatus for Three Dimensional Viewingof Images, issued to Douglas, U.S. Pat. No. 8,384,771, Method andApparatus for Three Dimensional Viewing of Images, issued to Douglas,Douglas, D. B., Petricoin, E. F., Liotta L., Wilson, E. D3D augmentedreality imaging system: proof of concept in mammography. Med Devices(Aucki), 2016; 9:277-83, Douglas, D. B., Boone, J. M., Petricoin, E.,Liotta, L., Wilson, E. Augmented Reality Imaging System: 3D Viewing of aBreast Cancer. J. Nat Sci. 2016; 2(9), and Douglas, D. B., Wilke, C. A.,Gibson, J. D., Boone, J. M., Wintermark, M. Augmented Reality: Advancesin Diagnostic Imaging. Multimodal Technologies and Interaction, 2017;1(4):29. In D3D imaging, the radiologist wears an augmented reality(AR), mixed reality (MR) or virtual reality (VR) headset and uses ajoystick or gaming controller. Advantages include improved depthperception and human machine interface. Still, there are severalchallenges faced with this approach. First, an area of interest (e.g.tumor) may be in close proximity to structures that are similar incomposition/density. Isolating the area of interest for betterexamination may be difficult. Second, many soft tissues in the body aremobile and deformable, so it can be difficult to achieve the bestorientation to properly compare the tumor at multiple time points.Efficiently aligning the orientation to do so may be difficult. Third,certain portions of a tumor can respond to treatment and decrease insize while other portions of a tumor demonstrate increases in size. Thepattern of tumor shrinkage has important prognostic implications.Furthermore, composition and complex morphologic features includingspiculations (spikes extending from the surface), irregular margins andenhancement also have important implications. Consequently, there is aneed for a system that facilitates recognition of the subtle, yetimportant changes in size, shape and margins. Fourth, a patient withmetastatic cancer has several areas of interest in different areas ofthe body. It is difficult and time consuming to find each of the areasof interest at every time point to determine interval change.Consequently, there is a need for a system that enables the observer todo this efficiently.

SUMMARY

All examples, aspects and features mentioned in this document can becombined in any technically possible way.

In accordance with an aspect of the invention a method comprises:generating a three-dimensional cursor that has a non-zero volume;responsive to a first input, moving the three-dimensional cursor withina three-dimensional image; responsive to a second input, selecting avolume of the three-dimensional image designated by thethree-dimensional cursor; and responsive to a third input, presenting amodified version of the selected volume of the three-dimensional image.In some implementations presenting the modified version of the selectedvolume of the three-dimensional image comprises removing an un-selectedvolume of the three-dimensional image from view. In some implementationspresenting the modified version of the selected volume of thethree-dimensional image comprises changing transparency of presentedtissues within the selected volume. In some implementations presentingthe modified version of the selected volume of the three-dimensionalimage comprises filtering a selected tissue to remove the selectedtissue from view. In some implementations presenting thethree-dimensional cursor with measurement markings on at least one edge,surface or side. In some implementations presenting the modified versionof the selected volume of the three-dimensional image comprisespresenting inputted location indicators. In some implementationspresenting the modified version of the selected volume of thethree-dimensional image comprises presenting inputted annotations. Someimplementations comprise changing a size dimension of thethree-dimensional cursor responsive to a fourth input. Someimplementations comprise changing a geometric shape of thethree-dimensional cursor responsive to a fifth input. Someimplementations comprise automatically generating a statisticalrepresentation of the selected volume of the three-dimensional image. Insome implementations presenting the modified version of the selectedvolume of the three-dimensional image comprises presenting at least onetissue type with false color. In some implementations presenting themodified version of the selected volume of the three-dimensional imagecomprises presenting volumetric changes over time with false color. Someimplementations comprise presenting multiple computed tomography imagesassociated with the selected volume using reference lines. Someimplementations comprise presenting multiple axial computed tomographyimages associated with the selected volume using reference lines. Someimplementations comprise presenting a maximum intensity projection (MIP)image of a positron emission tomography (PET) scan with thethree-dimensional cursor overlaid thereon to indicate orientation andlocation of the selected volume. Some implementations comprisepresenting a radiology report enhanced with information obtained usingthe three-dimensional cursor. Some implementations compriseautomatically calculating and presenting a quantitative analysis and aqualitative analysis associated with multiple time points. Someimplementations comprise presenting the modified version of the selectedvolume of the three-dimensional image comprises presenting inputtedregistration markers. Some implementations comprise automaticallycalculating volumetric change based on the registration markers. Someimplementations comprise automatically re-orienting the selected volumeof the three-dimensional image based on the registration markers. Someimplementations comprise using multiple volumes selected with thethree-dimensional cursor to designate a pre-operative planning pathwayfor guiding surgical intervention. Some implementations comprisepresenting the selected volume with an augmented reality, virtualreality or mixed reality headset.

In accordance with an aspect of the invention an apparatus comprises: acomputing device; and a human-machine interface comprising athree-dimensional cursor that has a non-zero volume; the human-machineinterface moving the three-dimensional cursor within a three-dimensionalimage responsive to a first input; the human-machine interface selectinga volume of the three-dimensional image designated by thethree-dimensional cursor responsive to a second input; and thehuman-machine interface presenting a modified version of the selectedvolume of the three-dimensional image responsive to a third input. Insome implementations, the human-machine interface removes an un-selectedvolume of the three-dimensional image from view. In someimplementations, the human-machine interface changes transparency ofpresented tissues within the selected volume. In some implementations,the human-machine interface filters a selected tissue to remove theselected tissue from view. In some implementations, the human-machineinterface presents the three-dimensional cursor with measurementmarkings on at least one edge, surface or side. In some implementations,the human-machine interface receives and implements inputted locationindicators. In some implementations, the human-machine interfacereceives and implements inputted annotations. In some implementations,the human-machine interface changes a size dimension of thethree-dimensional cursor responsive to a fourth input. In someimplementations, the human-machine interface changes a geometric shapeof the three-dimensional cursor responsive to a fifth input. In someimplementations, the human-machine interface automatically generates andpresents a statistical representation of the selected volume of thethree-dimensional image. In some implementations, the human-machineinterface presents at least one tissue type with false color. In someimplementations, the human-machine interface presents volumetric changesover time with false color. In some implementations, the human-machineinterface presents multiple computed tomography images associated withthe selected volume using reference lines. In some implementations, thehuman-machine interface presents multiple axial computed tomographyimages associated with the selected volume using reference lines. Insome implementations, the human-machine interface presents a maximumintensity projection (MIP) image of a positron emission tomography (PET)scan with the three-dimensional cursor overlaid thereon to indicateorientation and location of the selected volume. In someimplementations, the human-machine interface presents a radiology reportenhanced with information obtained using the three-dimensional cursor.In some implementations, the human-machine interface automaticallycalculates and presents a quantitative analysis and a qualitativeanalysis associated with multiple time points. In some implementations,the human-machine interface presents inputted registration markers. Insome implementations, the human-machine interface automaticallycalculates volumetric change after appropriate registration using theregistration markers. In some implementations, the human-machineinterface automatically re-orients the selected volume of thethree-dimensional image based on the registration markers. In someimplementations, the human-machine interface presents multiple volumesselected with the three-dimensional cursor to designate a pre-operativeplanning pathway for guiding surgical intervention. In someimplementations, the human-machine interface presents the selectedvolume with an augmented reality, virtual reality or mixed realityheadset.

BRIEF DESCRIPTION OF THE FIGURES

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1A illustrates a 3D cursor selecting a volume of interest from athree-dimensional medical image.

FIG. 1B illustrates the volume of interest selected with the 3D cursor;unselected portions have been removed from view.

FIG. 1C illustrates modification of the transparency of the selectedvolume of interest.

FIG. 1D illustrates filtering of selected areas of the selected volumeof interest.

FIG. 2 illustrates a variant of the 3D cursor of FIG. 1A withmeasurement markings on edges and sides.

FIG. 3 illustrates location indicators and annotations positionedrelative to the portion of the image within the selected volume ofinterest.

FIGS. 4A, 4B, and 4C illustrate three different examples of geometricshapes of the 3D cursor of FIG. 1A.

FIG. 5 illustrates presentation of a quantitative analysis of tissuesinside of the volume of interest selected with the 3D cursor of FIG. 1A.

FIG. 6 illustrates use of false color and transparency changes toenhance viewing of the selected volume of interest.

FIG. 7 illustrates association of multiple computed tomography (CT)images of the chest in lung windows with the interactive 3D cursor usingreference lines.

FIG. 8 illustrates association of multiple axial computed tomography(CT) slices of the chest in lung windows with the interactive 3D cursorusing reference lines.

FIG. 9 illustrates a maximum intensity projection (MIP) image of afludeoxyglucose (18F) positron emission tomography (PET) scan in whichtwo varying sized interactive 3D cursors are overlaid to indicate 3Dcursor shape, size, orientation, and location when respective volumes ofinterest were selected.

FIG. 10 illustrates a radiology report enhanced with informationobtained using the interactive 3D cursor and including quantitative andqualitative analysis.

FIG. 11 illustrates a radiology report enhanced with informationobtained using the interactive 3D cursor, and including addedquantitative and qualitative analysis at multiple time points.

FIGS. 12A, 12B and 12C illustrate a technique for correction formis-registration at multiple time points using three or more markers.

FIG. 13 illustrates use of multiple interactive 3D cursors to selectvolumes of interest to designate a safe pre-operative planning pathwayfor guiding surgical intervention.

FIG. 14 illustrates use of the interactive 3D cursor in an educationalsetting.

FIG. 15 illustrates process steps on a radiologist's review of apatient's image with integration of the interactive 3D cursor.

FIG. 16 illustrates a system for use of the interactive 3D cursor.

DETAILED DESCRIPTION

Some aspects, features and implementations described herein may includemachines such as computers, electronic components, radiologicalcomponents, optical components, and processes such ascomputer-implemented steps. It will be apparent to those of ordinaryskill in the art that the computer-implemented steps may be stored ascomputer-executable instructions on a non-transitory computer-readablemedium. Furthermore, it will be understood by those of ordinary skill inthe art that the computer-executable instructions may be executed on avariety of tangible processor devices. For ease of exposition, not everystep, device or component that may be part of a computer or data storagesystem is described herein. Those of ordinary skill in the art willrecognize such steps, devices and components in view of the teachings ofthe present disclosure and the knowledge generally available to those ofordinary skill in the art. The corresponding machines and processes aretherefore enabled and within the scope of the disclosure.

FIG. 1A illustrates a 3D (three-dimensional) cursor 100 overlaid on athree-dimensional medical image 102. In the illustrated example, the 3Dcursor 100 defines a cubic volume of interest. The medical image 102could include any portion of a body, or an entire body, for example andwithout limitation. For purposes of explanation the medical image 102includes different types of tissue. More specifically, the imageincludes a background material 104, such as fat, a lobulated mass 106, atubular-shaped vein 108, and an artery 110. The 3D cursor 100 can bemoved relative to the image, e.g. in three dimensions, such as bymanipulating an IO device such as a 3D mouse, for example and withoutlimitation. A button click or other input designates (selects) theportion of the image that is located inside the three-dimensional volumeof the 3D cursor 100. Distinguishing between a 3D image portion selectedby a 3D cursor and other unselected image portions is described in US2016/0026266 and U.S. Pat. No. 8,384,771, both of which are incorporatedby reference.

FIG. 1B illustrates the selected image portion of FIG. 1A. Moreparticularly, unselected portions of the image located outside of animage portion 112 selected with the 3D cursor 100 have been filtered-outor otherwise completely removed from view. Consequently, the removedportions of the image do not obstruct or hinder the view of the selectedimage portion. Moreover, the selected image portion 112 can bemanipulated and viewed as a separate and distinct image from the largermedical image 102 from which it was selected.

FIG. 1C illustrates modification of the transparency of the selectedimage portion 112. More specifically, transparency may be decreasedand/or increased such that tissues and other features can be betterobserved, e.g. such that overlapping tissues and features are visible.For example, tissues and features located proximate to the back of theselected image portion such as lobulated mass 106 can be seen throughoverlapping tissues and features located proximate to the front of theselected image portion such as vein 108, when transparency issufficiently increased. The transparency may be manipulated with the IOdevice to achieve various levels of transparency. Further, differentlevels of transparency may be applied to different portions of theselected image portion.

FIG. 1D illustrates filtering of selected areas or tissues of theselected image portion 112 to remove those areas or tissues from view.In the illustrated example the background material 104, vein 108, and anartery 110 have been removed from view, leaving only the lobulated mass106. The tissues to be filtered (removed from view) may be selectedbased on geometric shape, color, brightness, density, and any other of avariety of available image data, either alone or in combination.Moreover, a designated volume defined by a geometric shape may beremoved, e.g. a geometric shape that traverses tissue boundaries.

Transparency modification and tissue filtering facilitate presentationof certain tissue types of concern, both within the cursor and outsideof the cursor. Currently, the medical professional must see through anytissue within the cursor but external to the tissue type of concern fromthe viewing point of the medical professional, thus degrading thevisibility of the tissue of concern. The illustrated improvements enablethe medical professional to change the transparency of any tissue withinthe cursor-defined volume but external to the tissue type of concern.Alternatively, tissue types not of concern are subtracted from thevolume contained within the interactive 3D cursor, leaving only thetissue of concern in the presented image. Multiple interactive 3Dcursors in combination can be used to obtain varying patterns of tissuesubtraction. This helps to overcome the limitations of degradedvisibility due to tissue within the cursor but external to the tissuetype of concern from the viewing point of the medical professional.

FIG. 2 illustrates an implementation of the 3D cursor 100 withdimensional measurement markings. Dimensional measurement markings maybe available as a feature that can be turned ON and OFF. In theillustrated example, the 3D cursor is a 2 cm by 2 cm by 2 cm cube. Thedimensional measurement markings include tick marks 200, 202, and 204that respectively designate 1 mm, 5 mm, and 1 cm increments along theedges of the cube (and thus representing three dimensions). Tick marksthat represent different magnitudes may be uniquely represented tofacilitate visual size determination of the lobulated mass 106 thatrepresents the lesion of interest. 1 cm markings 206 are presented ineach of two dimensions on each side of the cube.

The dimensional measurement markings can help serve as a reference forradiologist's activities to include visual assessment, orientation,comparisons with prior scans or measurements. Advantages may includemitigating the current lack of metrics are available to the medicalprofessional to understand the size of the cursor and/or of the tissueelements contained within the cursor. This implementation placesmeasurement metrics on each edge and side of the cursor to help enablethe medical professional to rapidly understand the size of the subtendedvolume within the cursor. In the case where the cursor encapsulates avolume of concern such as a tumor, the three-dimensional size could berecorded in the medical professional report. This can help the visualassessment of each portion of the tumor to aid in the assessment ofsmall changes in size of findings including lobulations of a mass'smargin and spiculations.

Referring to FIG. 3, location indicators 300 and annotations 302 may beplaced by the radiologist or by automated techniques to highlightlocations or regions of concern within the interactive 3D cursor. Thelocation indicators may specify a point or region within the volume ofthe 3D cursor. Annotations can be added manually by the radiologist orby automated techniques to describe areas that are of concern, e.g.,growing, spiculation, irregular margin, indistinct margin, etc. Ifspiculations are on the surface of a tumor, this could be an indicatorof potential malignancy. The location indicators, such as, but notlimited to, arrow(s) pointing to key regions of interest within/outsidethe 3D cursor helps to overcome the limitation of the inability to markkey points within the cursor. This feature will be useful in discussionsbetween medical professions regarding a patient's condition. It willalso be useful in communicating imaging findings between a medicalprofessional and a patient.

Referring to FIGS. 4A, 4B, and 4C, the 3D cursor may be may beimplemented in a wide variety of different shapes. Examples include butare not limited to cube, cuboid, cylinder, sphere, ellipsoid, cone andtetrahedron. The shapes are not necessarily regular, and the lengths ofedges may be resized, e.g. overall geometric shape scaling or changingindividual edges, sides, or surfaces. For example, FIGS. 4A and 4Billustrate cuboid 3D cursors 400, 402 for which edge length has been setor selected based on the dimensions and orientation of the respectivefeature of interest 404, 406. FIG. 4C illustrates a spherical 3D cursor408 for which the diameter may be set or selected based on thedimensions of the feature of interest 410. In addition to dimensionalchanges, cursor geometric shape may be changed.

The ability to change the size, shape, and individual dimensions of the3D cursor enables the cursor to be customized based on the particularvolume of interest to the medical professional. A fixed-shape,fixed-size cursor might be too large or too small, e.g. so as to includea significant amount of tissue not of interest. For example, inexamining the lungs, placement of a cube-shaped cursor could cause ribsto be included in the image. Changing the shape of the 3D cursor wouldhelp to overcome this limitation. Customization could be accomplished bywide variety of techniques, possibly including but not limited toselecting an edge, side or vertex of the original 3D cursor with asecond type of cursor 412, and then “clicking and dragging” the selectededge, side, or vertex in the desired direction to expand or reduce thevolume of the original 3D cursor. The interface may also enableselection and change between multiple 3D geometric shapes, e.g. changingfrom cuboid to spherical. Scrolling on the conventional slices whilesimultaneously drawing shapes can also be performed to generate theprescribed 3D cursor volume. The interactive 3D cursor thus provides anefficient interface for tissue subtraction to provide enhancedvisualization of the tumor.

FIG. 5 illustrates presentation of a quantitative analysis 500 of alltissues inside a volume selected with the 3D cursor. The illustratedexample includes a bar graph but it is to be understood that any of awide variety of charts, graphs, and other techniques for presentation ofdata might be implemented. Quantitative analysis can help theradiologist understand how a feature of interest such as tumor 502(e.g., the lobulated mass 106, FIG. 1B) is changing in volume 504 overmultiple time points. The interface may include a statisticalrepresentation of the tissue types, possibly including but not limitedto a histogram bar chart to depict the volume (e.g., number of voxelsper unit volume) of the different types of tissue within the cursor,distinct markings for different types of tissue such as, but not limitedto, color coding the bars of the histogram bar chart.

FIG. 6 illustrates an implementation of the interactive 3D cursor 100with false color and transparency to enhance viewing. False color andtransparency may be dynamically adjusted and turned ON and OFF.Different false colors may be applied to different tissue types withinthe volume of the 3D cursor. The colors could be selected to correspondto the colors used in the statistical representation (FIG. 5).Alternatively, a respective unique false color could be selected foreach different tissue type, or tissue types of particular interest orconcern, and/or additional features of concern, e.g., irregular margin,indistinct margin, spiculation, etc. In the illustrated example, thebackground material 104 (fat) is depicted in light gray, the artery 110is depicted in red, the vein 108 is depicted in blue, and the lobulatedmass 106 is multicolored. Different colors may be selected or used toindicate stability of the lobulated mass 106 over time. For example,green may be used to indicate a stable volume 112 while orange is usedto denote a slow growth volume 114, thereby providing a visual warningindicator. Red may be used to indicate high rate of growth or concerningmargin volume 116. The extent of the volume of the lobulated mass can bedetermined automatically, e.g. based on density. Moreover, changes involume of sub-regions of the lobulated mass may also be automaticallydetermined, and color coding may be automatically implemented. This canhelp the radiologist understand how the mass is changing in volume overmultiple time points.

FIG. 7 illustrates association of multiple computed tomography (CT)images of the chest in lung windows with the interactive 3D cursor 100using reference lines 700. The illustrated example includes an axialimage 702, a sagittal image 704, and a coronal image 706 of the chest inlung windows. An advantage is enhanced ability to cross reference the 3Dcursor to the original 2D slices 702, 704, 706 from which total 3Dvolume was obtained. Medical professionals have experience andfamiliarity with 2D slices and may feel more confident in their findingsgiven the capability to switch back and forth between the 2D and 3Dvolumetric approaches. A small display adjacent to the interactive 3Dcursor could indicate which 2D slices contain tissue within in theinteractive 3D cursor. Then the medical professional could direct thesystem to automatically select those slices which have tissue within thecursor and display them on a nearby 2D display unit. A correspondingvisible boundary of the 3D cursor (e.g., red) projected on each of theslices may be presented.

FIG. 8 illustrates association of multiple axial computed tomography(CT) slices 800, 802, 804, 806 of the chest in lung windows with theinteractive 3D cursor 100 using reference lines 808. The multiple axialcomputed tomography (CT) slices of the chest in lung windows show thelocation of the 3D cursor, i.e. the slice area that includes across-section of the 3D cursor, which in the illustrated example hasselected a left upper lobe mass. Boundaries 810 of the 3D cursor in theslices are depicted in a color, e.g. red. Within the 3D cursor the lungcancer mass 106 is depicted in gray, surrounded by black that indicatesnon-cancerous lung tissue. This implementation helps the medicalprofessional to rapidly visualize where the interactive 3D cursor islocated relative to the slice images and the body. It also enables themedical professional to visualize the entire volumetric data with theinteractive 3D cursor accurately positioned within the volume.Transparency of tissue within the 3D volume could be changed so that theinteractive 3D cursor would stand out. This would help avoid left-rightorientation mistakes that might occur during treatment. Multipleinteractive 3D cursors which could be of differing sizes and/or shapescould be created and displayed.

FIG. 9 illustrates overlay of 3D cursors 100 a, 100 b on a maximumintensity projection (MIP) image 900 of a fludeoxyglucose (18F) positronemission tomography (PET) scan. Two different-sized interactive 3Dcursors are used to highlight two separate areas of concern, including3D cursor 100 a for a right lung mass and cursor 100 b for a vertebralbody metastasis. This helps to automatically transfer data (e.g.,picture of tissue within the cursor and statistical representations)from the viewing modality to the report of findings. Selection of keydata through human machine interface such as, but limited to, a screencapture can be automatically transferred to the report of findings. Thiswould provide quantitative results within the report together withqualitative impressions of the medical professional.

FIG. 10 illustrates a radiology report 1000 enhanced with informationobtained from the interactive 3D cursor. Qualitative findings 1002 andquantitative findings 1004 may be included along with patientidentifying information 1006, clinical history 1008, comparisons 1010,conclusions 1012, and recommendations 1014. Also included are a selectedvolume image 1016 and statistical graphic 1018. This helps toquantitatively track changes in volumes of concern (e.g., tumors) overtime.

FIG. 11 illustrates a radiology report 1100 enhanced with informationobtained from the interactive 3D cursor at multiple time points.Qualitative findings 1002 and quantitative findings 1004 may be includedalong with patient identifying information 1006, clinical history 1008,comparisons 1010, conclusions 1012, and recommendations 1014. Alsoincluded are selected volume images 1102, 1104 from different timepoints and respective statistical graphics 1106, 1108 from those timepoints. Follow up reports can include current and prior exams 1110, 1112with quantitative analysis and qualitative analysis on how the lesionhas changed over time. This may facilitate selection of a lesion (e.g.,tumor) at multiple time points using an interactive 3D cursor;qualitative assessment of the lesion at multiple time points; and,quantitative assessment of the lesion at multiple time points. Thiswould enable the medical professional to better assess how a particularlesion is changing over time. A report of current findings as outlinedin the previous implementation could be placed in a report together withthe data obtained from an earlier examination. This would enabletracking over time the progress of treatment or that of changes intissues of interest/concern.

FIGS. 12A, 12B, and 12C illustrate a registration technique by whichmis-registration can be corrected at multiple time points through theuse of three or more markers 12, 14, 16. Initially, the mass 106 withineach 3D cursor 100 image is noted using different locations within theinteractive 3D cursor and different orientations. Next, the user markssimilar locations on each image of the mass with registration markers.In the illustrated example, a yellow marker 12, a red marker 14, and ablue marker 16 correspond to the same respective parts of the mass oneach scan. Finally, tissues within the interactive 3D cursor are alignedin accordance with markers. Many soft tissues within the body can changein orientation from one scan to the next due to patient movement.Corresponding mis-registration can limit the ability to properly trackhow a lesion changes over time. This technique provides a method tocorrect for such mis-registration. Three or more recognizable spots ofthe lesion (e.g., tumor) can be marked with a false color, arrow, orother registration mark. Then, these locations can be automaticallyaligned with one another. Shadows can be added to help bring out depthperception. Proper alignment will accurately align the shadows. Thisenhances visual assessment for how a lesion is changing over time toinclude changes in tumor composition, size and morphology.

FIG. 13 illustrates use of multiple image volumes selected with the 3Dcursor to designate a safe pre-operative planning pathway to guidesurgical intervention. In the illustrated example, multiple greeninteractive 3D cursors 1300 mark a surgeon-selected dissection pathwaythat is deemed safe in the pre-operative setting. The interactive 3Dcursor 100 containing the cancerous lesion 106 is shown at a distal endof the planned surgical path represented by abutting or overlappingvolumes selected with the 3D cursors 1300. The selected path that thesurgeon will excise avoids the artery 110 with a minimum clearance of 10mm. This provides the advantage of 3D depiction of possible surgicalcuts. The path could include, but is not limited to, one or more of thefollowing properties: a serpentine shape; measurements couldsubsequently be made to measure absolute distance between a point on theplanned path to some region of concern (e.g., artery); the path couldalso be projected on a head mounted display at different intervalsduring the course of the operation. This feature would facilitatesurgical planning as well as a potential to improve accuracy of thesurgery.

FIG. 14 illustrates use of the interactive 3D cursor in an educationalsetting. Students 1400 are depicted wearing AR (augmented reality)headsets 1402 and an instructor 1404 is pointing to an abnormality onthe board 1406. This facilitates presentation of medical information(e.g., anatomy) in a classroom environment. The interactive 3D cursorcould be placed around the organ of interest and other parts of the bodycould be eliminated. Items from implementations discussed above such asmetrics and arrows could be used. The students would be provided 3D headdisplays and joined into a display system so that they could see thetissue within the interactive 3D cursor. This would eliminate anyconfusion on the part of the students as to what specific detail in theimagery was being discussed.

FIG. 15 illustrates process steps on a radiologist's review of apatient's image with integration of the interactive 3D cursor intohis/her practice. Step 1 is to create an interactive 3D cursor volumeand shape that approximates the size and shape of patient organ/tissuecorresponding to the item currently being inspected on the checklist.Step 2 is to position the interactive 3D cursor over the organ/tissue tobe inspected. The interactive 3D cursor as it is located within thetotal 3D image volume may be presented on a display. Step 3 is tosubtract from view all tissue external to the interactive 3D cursor. Theinteractive 3D cursor may be rotated to permit viewing from multipleangles. If interactive cursors are used at multiple time points to trackhow a particular lesion (e.g., tumor) changes over time, the 3D cursorscan be rotated in synchrony with on another. Step 4 is to generate astatistical representation e.g., a histogram of tissue densities—colorcoded with the types of tissue that are suspicious. Step 5 is tosubtract from view additional tissue within the interactive 3D cursor asdeemed appropriate by the medical professional. Step 6 is to inspect thevolume within the cursor and identify region(s) of interest and placeindicators, annotations, and registration markers relative to region(s)of interest. Step 7 is to extract a statistical representation andcapture imagery showing indicators, annotations, and registrationmarkers and residual tissue within the interactive 3D cursor to beinserted into the medical professional's report. Step 8 is to usecross-referencing as described the above to confirm findings. Step 9 isto iterate on the other items on the checklist until finished. Step 10is to prepare the report of the medical professional's findings. Thisprocedure provides an opportunity to enhance medical image reviewprocess by medical professionals.

FIG. 16 illustrates a system for use of the interactive 3D cursor. Amedical imaging device 1600 is connected to a computer workstation 1602.A wide variety of medical imaging devices and computer workstationscould be used. Images are captured by the medical imaging device andsent to the computer workstation. The computer workstation includesnon-volatile storage, computer-readable memory, processors, and avariety of other resources including but not limited to IO devices thatprovide a human-machine interface. In the illustrated example, the IOdevices include a monitor 1604, keyboard 1606, 3D mouse 1608, and VRheadset 1610. The IO devices are used to prompt a software program thatruns on the computer workstation to perform the various process stepsand implement the various features that have already been describedabove.

There are multiple potential advantages of the interactive 3D cursor.For example, there is reduction in time spent for classification ofmultiple lesions. The radiologist doesn't have to sort through manyprior imaging studies to find the lesion and the interactive 3D cursorwill save time. There is reduction in error when tracking multiplelesions, i.e. reducing the likelihood of mistakes when identifyingdifferent specific lesions that are nearby one another when comparingmultiple scans. One possibility is to analyze the images obtained usingthe 3D cursor and using multiple uniquely tagged (e.g. numbered) cursorsfor any suspicious regions. The medical profession could then switch toslices for confirmation.

Several features, aspects, embodiments and implementations have beendescribed. Nevertheless, it will be understood that a wide variety ofmodifications and combinations may be made without departing from thescope of the inventive concepts described herein. Accordingly, thosemodifications and combinations are within the scope of the followingclaims.

What is claimed is:
 1. A display unit, comprising: a processor; a lefteye display; a right eye display; a non-transitory memory havingcomputer-executable instructions stored thereupon, which when executedby the processor of the display unit, cause the display unit to performthe operations of: selecting a first left eye viewpoint for a left eyeof a user of a volume of interest containing a first item and a seconditem; selecting a first right eye viewpoint for a right eye of the userof the volume of interest containing the first item and the second item;selecting a first convergence point located at the first item for theleft eye of the user and the right eye of the user; displaying a firstleft eye image on the left eye display and a first right eye image onthe right eye display wherein the first left eye image is based on thefirst left eye viewpoint, the first convergence point and the volume ofinterest and wherein the first right eye image is based on the firstright eye viewpoint, the first convergence point and the volume ofinterest; performing filtering of the first item to create a filteredvolume of interest; selecting a second convergence point located at thesecond item for the left eye of the user and the right eye of the user;and displaying a second left eye image on the left eye display and asecond right eye image on the right eye display wherein the second lefteye image is based on the first left eye viewpoint, the secondconvergence point and the filtered volume of interest and wherein thesecond right eye image is based on the first right eye viewpoint, thesecond convergence point and the filtered volume of interest.
 2. Thedisplay unit of claim 1, further comprising wherein the first left eyeviewpoint and first right eye viewpoint are separated by an interoculardistance.
 3. The display unit of claim 1, wherein the display unit isfurther configured to perform filtering of the first item by voxelproperty to create the filtered volume of interest.
 4. The display unitof claim 1, wherein the display unit is further configured to performfiltering of the first item by the geometric boundary method to createthe filtered volume of interest.
 5. The display unit of claim 1, whereinthe display unit is further configured to perform zooming in on thefiltered volume of interest by performing the operations of: generatinga second left eye viewpoint wherein the second left eye viewpoint iscloser to the filtered volume of interest than the first left eyeviewpoint; generating a second right eye viewpoint wherein the secondright eye viewpoint is closer to the filtered volume of interest thanthe first right eye viewpoint; and displaying a third left eye image onthe left eye display and a third right eye image on the right eyedisplay wherein the third left eye image is based on the second left eyeviewpoint, the second convergence point and the filtered volume ofinterest and wherein the third right eye image is based on the secondright eye viewpoint, the second convergence point and the filteredvolume of interest.
 6. The display unit of claim 1, wherein the displayunit is further configured to perform rotating the filtered volume ofinterest.
 7. The display unit of claim 1, wherein the display unit isfurther configured to perform coloring the first item and the seconditem.
 8. A display unit, comprising: a left eye display; a right eyedisplay; and a communications interface wherein the communicationsinterface is in communication with a non-transitory memory and aprocessor wherein the non-transitory memory having computer-executableinstructions, which when executed by the processor, perform theoperations of: selecting a first left eye viewpoint for a left eye of auser of a volume of interest containing a first item and a second item;selecting a first right eye viewpoint for a right eye of the user of thevolume of interest containing the first item and the second item;selecting a first convergence point located at the first item for theleft eye of the user and the right eye of the user; displaying a firstleft eye image on the left eye display and a first right eye image onthe right eye display wherein the first left eye image is based on thefirst left eye viewpoint, the first convergence point and the volume ofinterest and wherein the first right eye image is based on the firstright eye viewpoint, the first convergence point and the volume ofinterest; performing filtering of the first item to create a filteredvolume of interest; selecting a second convergence point located at thesecond item for the left eye of the user and the right eye of the user;and displaying a second left eye image on the left eye display and asecond right eye image on the right eye display wherein the second lefteye image is based on the first left eye viewpoint, the secondconvergence point and the filtered volume of interest and wherein thesecond right eye image is based on the first right eye viewpoint, thesecond convergence point and the filtered volume of interest.
 9. Thedisplay unit of claim 8, further comprising wherein the first left eyeviewpoint and first right eye viewpoint are separated by an interoculardistance.
 10. The display unit of claim 8, further comprising theprocessor configured to perform filtering of the first item by voxelproperty to create the filtered volume of interest.
 11. The display unitof claim 8, further comprising the processor configured to performfiltering of the first item by the geometric boundary method to createthe filtered volume of interest.
 12. The display unit of claim 8,further comprising the processor configured to zoom in on the filteredvolume of interest by performing the operations of: generating a secondleft eye viewpoint wherein the second left eye viewpoint is closer tothe filtered volume of interest than the first left eye viewpoint;generating a second right eye viewpoint wherein the second right eyeviewpoint is closer to the filtered volume of interest than the firstright eye viewpoint; and displaying a third left eye image on the lefteye display and a third right eye image on the right eye display whereinthe third left eye image is based on the second left eye viewpoint, thesecond convergence point and the filtered volume of interest and whereinthe third right eye image is based on the second right eye viewpoint,the second convergence point and the filtered volume of interest. 13.The display unit of claim 8, further comprising the processor configuredto rotate the filtered volume of interest.
 14. The display unit of claim8, further comprising the processor configured to color the first itemand the second item.
 15. A non-transitory computer storage medium havingcomputer-executable instructions which, when executed by a computingdevice, cause the computing device to perform the operations of:selecting a first left eye viewpoint for a left eye of a user of avolume of interest containing a first item and a second item; selectinga first right eye viewpoint for a right eye of the user of the volume ofinterest containing the first item and the second item; selecting afirst convergence point located at the first item for the left eye ofthe user and the right eye of the user; displaying a first left eyeimage on the left eye display and a first right eye image on the righteye display wherein the first left eye image is based on the first lefteye viewpoint, the first convergence point and the volume of interestand wherein the first right eye image is based on the first right eyeviewpoint, the first convergence point and the volume of interest;performing filtering of the first item to create a filtered volume ofinterest; selecting a second convergence point located at the seconditem for the left eye of the user and the right eye of the user; anddisplaying a second left eye image on the left eye display and a secondright eye image on the right eye display wherein the second left eyeimage is based on the first left eye viewpoint, the second convergencepoint and the filtered volume of interest and wherein the second righteye image is based on the first right eye viewpoint, the secondconvergence point and the filtered volume of interest.
 16. Thenon-transitory computer storage medium of claim 15, further comprisingwherein the first left eye viewpoint and first right eye viewpoint areseparated by an interocular distance.
 17. The non-transitory computerstorage medium of claim 15, further comprising performing filtering ofthe first item by voxel property to create the filtered volume ofinterest.
 18. The non-transitory computer storage medium of claim 15,further comprising performing filtering of the first item by thegeometric boundary method to create the filtered volume of interest. 19.The non-transitory computer storage medium of claim 15, furthercomprising zooming in on the filtered volume of interest by performingthe operations of: generating a second left eye viewpoint wherein thesecond left eye viewpoint is closer to the filtered volume of interestthan the first left eye viewpoint; generating a second right eyeviewpoint wherein the second right eye viewpoint is closer to thefiltered volume of interest than the first right eye viewpoint; anddisplaying a third left eye image on the left eye display and a thirdright eye image on the right eye display wherein the third left eyeimage is based on the second left eye viewpoint, the second convergencepoint and the filtered volume of interest and wherein the third righteye image is based on the second right eye viewpoint, the secondconvergence point and the filtered volume of interest.
 20. Thenon-transitory computer storage medium of claim 15, further comprisingrotating the filtered volume of interest.
 21. The non-transitorycomputer storage medium of claim 15, further comprising coloring thefirst item and the second item.
 22. A method comprising: selecting afirst left eye viewpoint for a left eye of a user of a volume ofinterest containing a first item and a second item; selecting a firstright eye viewpoint for a right eye of the user of the volume ofinterest containing the first item and the second item; selecting afirst convergence point located at the first item for the left eye ofthe user and the right eye of the user; displaying a first left eyeimage on a left eye display and a first right eye image on a right eyedisplay wherein the first left eye image is based on the first left eyeviewpoint, the first convergence point and the volume of interest andwherein the first right eye image is based on the first right eyeviewpoint, the first convergence point and the volume of interest;performing filtering of the first item to create a filtered volume ofinterest; selecting a second convergence point located at the seconditem for the left eye of the user and the right eye of the user; anddisplaying a second left eye image on the left eye display and a secondright eye image on the right eye display wherein the second left eyeimage is based on the first left eye viewpoint, the second convergencepoint and the filtered volume of interest and wherein the second righteye image is based on the first right eye viewpoint, the secondconvergence point and the filtered volume of interest.
 23. The method ofclaim 22, further comprising wherein the first left eye viewpoint andfirst right eye viewpoint are separated by an interocular distance. 24.The method of claim 22, further comprising performing filtering of thefirst item by voxel property to create the filtered volume of interest.25. The method of claim 22, further comprising performing filtering ofthe first item by the geometric boundary method to create the filteredvolume of interest.
 26. The method of claim 22, further comprisingzooming in on the filtered volume of interest by performing theoperations of: generating a second left eye viewpoint wherein the secondleft eye viewpoint is closer to the filtered volume of interest than thefirst left eye viewpoint; generating a second right eye viewpointwherein the second right eye viewpoint is closer to the filtered volumeof interest than the first right eye viewpoint; and displaying a thirdleft eye image on the left eye display and a third right eye image onthe right eye display wherein the third left eye image is based on thesecond left eye viewpoint, the second convergence point and the filteredvolume of interest and wherein the third right eye image is based on thesecond right eye viewpoint, the second convergence point and thefiltered volume of interest.
 27. The method of claim 22, furthercomprising rotating the filtered volume of interest.
 28. The method ofclaim 22, further comprising coloring the first item and the seconditem.